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METHOD OF OPERATING A DISPLAY SYSTEM IN A VEHICLE 



FIELD OF THE INVENTION 

[0001] The invention relates to display systems for vehicles, having a camera, an image 

processing unit, a display screen positioned inside the vehicle, and a sensor system for detecting 
the movement data of the vehicle, are known from the related art. 

BACKGROUND OF THE INVENTION 

[0002] German Patent Application 100 45 616 Al describes a display system of this type 

which is to assist the driver, particularly during parking. 

[0003] A large problem in the use of known display systems is that the camera provided 

for recording moves with the vehicle. I.e., the driver is shown live images on the display screen 
which correspond to the current position of the camera, which is positioned, for example, on the 

1 



rear of the vehicle. Due to this continuous change of the camera viewpoint together with the 
vehicle, the mterpretation of the images displayed is made more diflScult for the driver, through 
which the assistance of the driver by display systems according to this type, particularly during 
parking, is significantly restricted. 

5 SUMMARY OF THE INVENTION 

[0004] The object of the present invention is therefore to provide a method of operating a 

display system in a vehicle, through which the interpretation of the images displayed is made 
easier for the driver. 

[0005] According to the invention, a method of operating a display system is provided in a 

10 vehicle is having at least one camera. The camera can be used for recording an observation region 
in the surroundings of the vehicle. The system has an image processing unit, in which the image 
data coming fi^om the camera may be processed and a display screen inside the vehicle, on which 
the image data coming from the image processing unit may be displayed as an image. A sensor 
system is employed for detecting vehicle movement and relaying as movement data to the image 
15 processing unit. The method includes the steps: 

at a starting instant, a vehicle position is detennined as a starting point; 
the vehicle is moved further, beginning fi-om the starting point; 

the rotational angle of the vehicle around its vertical axis is detected by the sensor system, 
beginning from the orientation at the starting point; 
20 the image data coming fi-om the camera is processed in the image processing unit, taking 
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into consideration the respective current rotational angle of the vehicle, in such a way that the 
orientation of objects from the observation region displayed in the image on the display screen 
corresponds to the orientation at the starting instant. 

[0006] The present invention is based on the basic idea that the driver is no longer 

S displayed the current live images recorded by the camera, but rather the image data of these live 
images is suitably preprocessed in an image processing unit. The preprocessing of the image data 
in the image processing unit is used in this case so that the orientation of the objects displayed on 
the display screen, particularly marking lines, curbstones, or other vehicles, always corresponds to 
the orientation at the starting instant. In other words, this means that the driver has the ability, 

10 through the achievement of the method according to the present invention, to always have the 
image data recorded by the camera displayed in a rotation-free representation. According to the 
present invention, the rotational angle of the motor vehicle around its vertical axis, beginning from 
the orientation in the starting point, is detected by a suitable sensor system, for example a 
steering-angle sensor in combination with a speed sensor and a timekeeping unit, and the image 

IS data recorded by the camera is processed in the image processing unit in such a way that the 

images displayed are always rotated diametrically opposed by precisely the angle around which 
the vehicle has rotated, beginning from the starting point. 

[0007] As a result, through the method according to the present invention, the driver is 

displayed images on the display screen which correspond to a fixed observation point outside the 
20 vehicle. 



3 



[0008] In order to be able to display the largest possible observation region in the 

surroundings of the vehicle, it is particularly advantageous if the image data of multiple cameras 
may be mixed together in the image processing unit and combined into a joint image. For 
example, it is conceivable for two video cameras, particularly CCD cameras, to be positioned on 
5 the rear of the vehicle, one camera observing the region left of center and the center and the other 
camera observing the region right of center and the center. The image data of both cameras may 
then be mixed in order to be able to display one single image having a correspondingly enlarged 
observation region. 

[0009] Alternatively to this method and/or as an addition thereto, wide angle lenses may 

10 be provided on the cameras, in order to enlarge the observation region. Since distortion of 
straight lines is produced by the wide angle lenses, which in turn makes interpretation of the 
images displayed more difficult for the driver, a processmg step may also be performed in the 
image processing imit, through which the distortions produced by the wide angle lenses may be 
removed again using computers. 

15 [0010] In addition to the derotation of the images displayed on the display screen 

accordmg to the present invention, the mterpretation may also be made easier for the driver if the 
image data is processed in the image processing unit in such a way that the image displayed on the 
display screen corresponds to a perspective outside the vehicle, particularly a top view fi-om 
above the vehicle. Suitable methods for converting an image into a top view are known, for 

20 example, from German Patent 197 41 896. By combining the derotation according to the present 
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invention and the display from the perspective above the vehicle, the driver obtains images 
displayed on the display screen which correspond to a fixed observation point above the vehicle. 
This display method is extremely compliant with the human visualization ability, so that the driver 
receives a good orientation aid, particularly during backwards parking. 

[0011] In the display systems according to this type (vehicle display systems), the 

observation region recordable by the camera is, as a rule, essentially rectangular, the ratio of 
height and width of the observation region corresponding to the ratio of height and width of the 
display screen, so that normally the entire observation region may be displayed on the display 
screen. If, according to the present invention, the image data is displayed on the display screen in 
such a way that the observation region is displayed so that the rotation of the vehicle around its 
vertical axis is removed using computers, it follows that the size of the displayable observation 
region necessarily no longer corresponds to the size of the display screen. This is because, 
through the method according to the present invention, the displayed observation region must be 
pivoted around the angle around which the vehicle has rotated around the vertical axis since the 
starting instant. As a function of the requirements of the respective driving situation, it is 
therefore necessary to suitably reduce the observation region displayed on the display screen by a 
reduction factor. 

[0012] In particular, it is conceivable that the reduction factor is selected as a function of 

the rotational angle of the vehicle smce the starting instant in such a way that the entire 
observation region of the camera may be displayed on the display screen. In this case, the entire 



display screen is necessarily not used to display the observation region, the displayed observation 
region simultaneously being reduced. With this procedure, however, it is always ensured that the 
driver is displayed all image information which is recorded by the cameras. 

[0013] As an alternative to displaying the entire observation region recordable by the 

camera, it is also conceivable for other reduction factors to be selected as a function of specific 
parameters, for example, the vehicle speed or the respective steering angle, so that only a slight 
reduction is necessary in this case, however, the comers of the observation region recordable by 
the camera not being displayed on the display screen. 

[0014] In specific driving situations, image information fi-om very specific regions of the 

observation region is of particular interest to the driver. According to the present invention, 
specific image details from the observation region may be zoomed in on during display on the 
display screen by selecting a corresponding zoom factor. I. e., depending on the selection of the 
zoom factor, the entire observation region is no longer displayed on the display screen, but rather 
an enlarged detail therefrom. 

[0015] According to a preferred embodiment, it is also possible in this case to change the 

position of the zoomed image detail in relation to the boundaries of the observation region. I. e., 
the zoomed image detail may be displaced forward, back, right, and left in relation to the 
boundaries of the observation region, so that as a result, an image detail which is arbitrarily 
selectable in its size and position is displayable. 



[0016] According to the present invention, the way in which the zoom factor is selected is 

basically arbitrary. According to a first embodiment of the present invention, the operator has the 
ability to manually set the zoom factor and/or the position of the zoomed image detail 1. e., 
starting from the respective image displayed, the driver may zoom in and out on the image 
displayed by operating corresponding suitable operating elements, for example, pushbuttons, and 
in this way additionally displace the image detail displayed forward, back, left, or right within the 
boundaries of the observation region. 

[0017] As an alternative to manual selection of the zoom factor, it is also conceivable that 

the zoom factor and/or the position of the zoomed image detail is performed automatically by the 
image processing unit as a function of other state parameters. 

[0018] The method according to the present invention is described for exemplary 

purposes in the following with reference to the drawing. The various features of novelty which 
characterize the invention are pointed out with particularity in the clauns annexed to and forming 
a part of this disclosure. For a better understanding of the mvention, its operating advantages 
and specific objects attained by its uses, reference is made to the accompanying drawings and 
descriptive matter in which preferred embodiments of the invention are illustrated. 



BRIEF DESCRIPTION OF THE DRAWINGS 

[0019] Fig. 1 is a schematic illustration of a vehicle in a first driving situation (starting 

instant) in a view from above; 



[0020] Fig. 2 is an image which is recorded in the driving situation shown in Fig. 1 

displayed on the display screen in the vehicle; 

[0021] Fig. 3 is a view from above the vehicle shown in Fig. 1, in a second driving 

situation; 

[0022] Fig. 4 is the image which is recorded in the driving situation shown in Fig. 3 

displayed on the display screen; 

[0023] Fig. 5 is a view from above from the vehicle shown in Fig. 1 in a third driving 

situation in; 

[0024] Fig. 6 is the image which is recorded in the driving situation shown in Fig. S 

displayed on the display screen; 

[0025] Fig. 7 is a view showing the same driving situation as shown in Fig. S, with an 

additional image detail in the observation region for zoom display; 

[0026] Fig. 8 is the image which is recorded in the driving situation shown in Fig. 7 

displayed on the display screen in the zoomed image detail. 
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DESCRIPTION OF THE PREFERRED EMBODIMENT 

[0027] Referring to the drawings in particular. Fig. 1 illustrates a vehicle 01 in a first 

driving situation, specifically while looking for a parking place, in a view fi-om above. Vehicle 01 
is equipped with two cameras, which are positioned on the rear of the vehicle and using which an 
observation region 02 behind vehicle 01, indicated by dashed lines in Fig. 1, may be recorded. 
Multiple parking bays (parking spots) 03 are located to the left of vehicle 01, which are delimited 
by marking stripes 04 applied to the ground. 

[0028] If the driver now wishes to park vehicle 01 in a parking bay 03a, the driver is 

assisted in the way according to the present invention by the display system provided on the 
vehicle 01, having cameras, unage processing unit, display screen inside the vehicle, and sensor 
system for detecting the vehicle movement data. The driving situation illustrated in Fig. 1 
represents the starting instant of the parking procedure, in which the vehicle assumes a position 
which corresponds to the desired starting point. 

[0029] A display screen 05 is schematically illustrated in Fig. 2, on which observation 

region 02 is displayed in the starting mstant by an unage 07. At the upper edge of display screen 
05, the rear of vehicle 01 is inserted into the live image as a schematic symbol 06. The other 
objects displayed on display screen 05 correspond to the situation which is recorded by the 
cameras at the rear of the vehicle at the corresponding instant. In particular, marking stripes 04a 
and 04b, which delimit parking bay 03a to be steered into and whose ends extend into observation 
region 02 (see Fig. 1), are displayed to the driver on display screen 05. If the driver now wishes 



to park in parking bay 03a delimited by marking stripes 04a and 04b, the driver initiates the image 
processing method according to the present invention by operating a suitable operating element, 
for example a pushbutton. 

[0030] In Fig. 3, a second driving situation with vehicle 01 is illustrated, in which vehicle 

01, beginning from the first driving situation (see Fig. 1), has already approached parking bay 03a. 
During the vehicle movement to approach parking bay 03a, the vehicle has rotated by 
approximately 45' around its vertical axis, so that a correspondingly changed part of the 
surrounding region is recorded in observation region 02 by the cameras. 

[0031] In Fig. 4, display screen OS is shown while displaying the second driving situation 

(see Fig. 3). It may be seen that the orientation of marking stripes 04a and 04b relative to display 
screen OS corresponds to the orientation at the starting instant (see Fig. 2), in spite of the intrmsic 
movement of vehicle 01. I. e., marking stripes 04a and 04b still run, in spite of the intrinsic 
movement of vehicle 01, essentially horizontally in image 07, which corresponds to the orientation 
of marking stripes 04a and 04b at the starting instant. During the processing of the image data 
recorded by the cameras in the image processing unit, the image data is converted in such a way 
that a perspective which corresponds to a top view from above the vehicle is displayed for the 
driver. In addition, distortions which arise due to the wide angle lenses provided on the cameras 
are removed using computers, so that straight markmg stripes 04 are also displayed as straight 
lines on display screen OS. 
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[0032] In order that entire observation region 02 may also be displayed in the second 

driving situation (see Fig. 3), in which observation region 02 has pivoted by approximately 45' in 
relation to the fixed objects in the vehicle surroundings, image 07 displayed on display screen 05 
is correspondingly reduced, I. e,, image 07 no longer fills up entire display screen 05, but rather is 
enclosed by a blackened display screen region 08. This effect of incomplete use of display screen 
05 during use of the method according to the present invention may be avoided by using circular 
display screens and selecting a circular observation region. 

[0033] Fig. 5 illustrates vehicle 01 in a third driving situation, in which vehicle 01 has 

already completely reached parking bay 03 a and must only move straight backward to arrive at 
the final position. The ends of marking stripes 04a and 04b have ah-eady moved out of 
observation region 02 and therefore may no longer be displayed on display screen 05. 

[0034] Fig. 6 illustrates the display of image 07 on display screen 05 during the third 

driving situation (see Fig. 5). Marking stripes 04a and 04b again run horizontally over image 
screen 05. Since observation region 02 is now positioned perpendicularly to the original 
orientation in the starting point, the reduction factor for reducing image 07 may be increased 
again to in comparison to the second driving situation, since now the entire height of image 05 is 
available for displaying image 07. 

[0035] In Fig. 7, the third driving situation corresponding to Fig. 5 is illustrated once 

again, the driver having selected an image detail 09 in observation 15 region 02 to be zoomed in 
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on by operating corresponding operating elements. During the selection of image detail 09, the 
driver may determine the size of image detail 09 and its relative position to the boundaries of 
observation region 02 to the front, back, right, and left. 

[0036] Fig. 8 shows display screen 05 while displaying the image data from zoomed 

5 image detail 09. Symbol 06 for indicating vehicle 01 in display screen 05 is enlarged in 

accordance with the zoom factor selected. By operating buttons 10 and 1 1, the driver may 
increase and/or reduce the zoom factor, in order to change the size of image detail 09 to be 
displayed on display screen 05. By operating buttons 12, 13, 14, and 15, the driver may displace 
zoomed image detail 09 to the front, back, right, and left within the boundaries of observation 
10 region 02. As a result, the driver has the ability to have unage detail 09 from observation region 
02 which he requires for dealing with the current driving situation, for 30 example during 
backward parking, displayed on display screen 05 at any time. Parts of the observation region 
which are not of interest to the driver may be hidden in this way. The zooming for displaying 
image detail 09 is not performed by adjusting the focus of the camera lens, but by selecting the 
1 5 image data in the imag^ processing unit. 

[0037] While specific embodiments of the invention have been shown and described in 

detail to illustrate the application of the principles of the invention, it will be understood that the 
invention may be embodied otherwise without departing from such principles. 
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